Advertisement Tracking

ABSTRACT

Methods, systems, and computer-readable media for determining the efficacy of an advertisement are described herein. A computing device may receive an advertisement from an advertisement server. The computing device may determine advertisement information associated with the presentation of the advertisement. The advertisement information may be sent to the advertisement server.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 15/397,329, filed on Jan. 3, 2017, entitled“ADVERTISEMENT TRACKING”, which claims priority to U.S. patentapplication Ser. No. 13/795,919, filed Mar. 12, 2013. These applicationsare incorporated herein by reference.

BACKGROUND

Advertisers in many industries have long sought to determine theefficacy of their advertisements and to make their advertisements moreeffective on viewers. These advertisers pay millions of dollars to havetheir ads presented during consumption of audio and/or video content, inthe hopes that their ads are noticed by the user. There is anever-present need, however, to help these advertisers in gauging theeffectiveness of their ads.

BRIEF SUMMARY

The following presents a simplified summary of the disclosure in orderto provide a basic understanding of some aspects. It is not intended toidentify key or critical elements of the disclosure or to delineate thescope thereof. The following summary merely presents some concepts ofthe disclosure in a simplified form as a prelude to the more detaileddescription provided below.

According to one or more aspects, a display device which shows videocontent with advertisement portions may monitor user actions during theadvertisement to determine the efficacy of the advertisement. Electronicadvertisements may be presented to users via various transmission media,such as a digital video multicast or broadcasts, video-on-demand,Internet web sites, or other software applications. During thepresentation of an advertisement, the device may monitor many possibleactions a user may take such as, changing the audio level of theadvertisement during play, obstructing the advertisement, or changingthe z-order (e.g., the order determining which application remainsvisible when two applications overlap in the display area) of the windowcontaining the advertisement. The display device may track all theseuser actions and write them to an advertisement efficacy file reportalong with the time of the action. That report can then be sent to anadvertisement server for further processing to determine the advertisingeffectiveness.

According to additional aspects, a display device which shows a videocontent with advertisement portions may restrict user action during anadvertisement via instructions in a configuration file sent from acontent server to the display device with the advertisement. Theserestrictions may include a prohibition on muting the advertisement orprohibiting the lessening of the audio level of the advertisement past acertain predetermined level, ensuring that the advertisement (or apredetermined portion of the advertisement) display is not obscured byother onscreen interfaces, restricting resizing of the advertisement, orother options described herein. Advertisers may dictate the instructionsto be placed in the configuration file based on certain optionsindicated by the content server that serve the requirements of theadvertiser and the display device.

According to additional aspects, a display device that displays orpresents video content with advertisement portions may respond to a useraction with another action. For example, if a user lessens the audiolevel of an advertisement during play, the device may respond byactivating closed-captioning for the duration of the advertisement.Alternatively, the device may pause the video content until the userraises the audio level to a certain level. The various user interactionlevels may also be included in the report to the advertisers.

The details of these and other embodiments of the present disclosure areset forth in the accompanying drawings and the description below. Otherfeatures and advantages of the disclosure will be apparent from thedescription and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present disclosure and theadvantages thereof may be acquired by referring to the followingdescription in consideration of the accompanying drawings, in which likereference numbers indicate like features, and wherein:

FIG. 1 illustrates an example system in which various features discussedherein may be implemented;

FIG. 2 illustrates an example computing device and network environmentthat may be used to implement one or more aspects of the disclosure;

FIG. 3 illustrates a display area playing an advertisement, obscured bya window containing a chat program;

FIG. 4 illustrates a display area playing an advertisement, such thatthe audio level has been lessened;

FIG. 5 is a flow diagram illustrating an example advertisement efficacyfile report creation process, according to one or more aspects of thedisclosure;

FIG. 6 is a flow diagram illustrating an example process of userrestriction during an advertisement, according to one or more aspects ofthe disclosure;

FIG. 7 is a flow diagram illustrating actions of the display device inresponse to user action during an advertisement, according to one ormore aspects of the disclosure;

FIG. 8 illustrates a display area playing an advertisement, such thatthe audio level has been lessened, and in response, the device activatesclosed captioning;

FIG. 9 illustrates a display area playing an advertisement, obscured bya window containing a chat program, and in response, the device pausesthe video content;

FIGS. 10a and 10b illustrate embodiments of configuration files used toinstruct the display device; and

FIG. 11 is a flow diagram illustrating actions of the advertisementserver, according to one or more aspects of the disclosure.

DETAILED DESCRIPTION

As will by one of skill in the art upon reading the followingdisclosure, various aspects described herein may be appreciated embodiedas a method, a computer system, or a computer program product.Accordingly, those aspects may take the form of an entirely hardwareembodiment, an entirely software embodiment or an embodiment combiningsoftware and hardware aspects. Furthermore, such aspects may take theform of a computer program product stored by one or morecomputer-readable storage media having computer-readable program code,or instructions, embodied in or on the storage media. Any suitablecomputer readable storage media may be utilized, including hard disks,CD-ROMs, optical storage devices, removable storage media, solid statememory, RAM, magnetic storage devices, and/or any combination thereof.In addition, the functionality may be embodied in whole or in part infirmware or hardware equivalents, such as integrated circuits, fieldprogrammable gate arrays (FPGAs), and the like. Various signalsrepresenting data or events as described herein may be transferredbetween a source and a destination in the form of electromagnetic wavestraveling through signal-conducting media such as metal wires, opticalfibers, and/or wireless transmission media (e.g., air and/or space).

FIG. 1 illustrates an example communication network 100 on which many ofthe various features described herein may be implemented. Network 100may be any type of information distribution network, such as satellite,telephone, cellular, wireless, etc. One example may be an optical fibernetwork, a coaxial cable network, or a hybrid fiber/coax distributionnetwork. Such networks 100 use a series of interconnected communicationlinks 101 (e.g., coaxial cables, optical fibers, wireless, etc.) toconnect multiple premises 102 (e.g., businesses, homes, consumerdwellings, etc.) to a local office or headend 103. The local office 103may transmit downstream information signals onto the links 101, and eachpremises 102 may have a receiver used to receive and process thosesignals.

There may be one link 101 originating from the local office 103, and itmay be split a number of times to distribute the signal to variouspremises 102 in the vicinity (which may be many miles) of the localoffice 103. The links 101 may include components not illustrated, suchas splitters, filters, amplifiers, etc. to help convey the signalclearly, but in general each split introduces a bit of signaldegradation. Portions of the links 101 may also be implemented withfiber-optic cable, while other portions may be implemented with coaxialcable, other lines, or wireless communication paths. By running fiberoptic cable along some portions, for example, signal degradation may besignificantly minimized, allowing a single local office 103 to reacheven farther with its network of links 101 than before.

The local office 103 may include an interface, such as a terminationsystem (TS) 104. More specifically, the interface 104 may be a cablemodem termination system (CMTS), which may be a computing deviceconfigured to manage communications between devices on the network oflinks 101 and backend devices such as servers 105-107 (to be discussedfurther below). The interface 104 may be as specified in a standard,such as the Data Over Cable Service Interface Specification (DOCSIS)standard, published by Cable Television Laboratories, Inc. (a.k.a.CableLabs), or it may be a similar or modified device instead. Theinterface 104 may be configured to place data on one or more downstreamfrequencies to be received by modems at the various premises 102, and toreceive upstream communications from those modems on one or moreupstream frequencies.

The local office 103 may also include one or more network interfaces108, which can permit the local office 103 to communicate with variousother external networks 109. These networks 109 may include, forexample, networks of Internet devices, telephone networks, cellulartelephone networks, fiber optic networks, local wireless networks (e.g.,WiMAX), satellite networks, and any other desired network, and thenetwork interface 108 may include the corresponding circuitry needed tocommunicate on the external networks 109, and to other devices on thenetwork such as a cellular telephone network and its corresponding cellphones.

As noted above, the local office 103 may include a variety of servers105-107 that may be configured to perform various functions. Forexample, the local office 103 may include a push notification server105. The push notification server 105 may generate push notifications todeliver data and/or commands to the various premises 102 in the network(or more specifically, to the devices in the premises 102 that areconfigured to detect such notifications). The local office 103 may alsoinclude a content server 106. The content server 106 may be one or morecomputing devices that are configured to provide content to users attheir premises. This content may be, for example, video on demandmovies, television programs, songs, text listings, etc. The contentserver 106 may include software to validate user identities andentitlements, to locate and retrieve requested content, to encrypt thecontent, and to initiate delivery (e.g., streaming) of the content tothe requesting user(s) and/or device(s).

The local office 103 may also include one or more application servers107. An application server 107 may be a computing device configured tooffer any desired service, and may run various languages and operatingsystems (e.g., servlets and JSP pages running on Tomcat/MySQL, OSX, BSD,Ubuntu, Redhat, HTML5, JavaScript, AJAX and COMET). For example, anapplication server may be responsible for collecting television programlistings information and generating a data download for electronicprogram guide listings. Another application server may be responsiblefor monitoring user viewing habits and collecting that information foruse in selecting advertisements. Yet another application server may beresponsible for formatting and inserting advertisements in a videostream being transmitted to the premises 102. Although shown separately,one of ordinary skill in the art will appreciate that the push server105, content server 106, and application server 107 may be combined.Further, here the push server 105, content server 106, and applicationserver 107 are shown generally, and it will be understood that they mayeach contain memory storing computer executable instructions to cause aprocessor to perform steps described herein and/or memory for storingdata.

An example premises 102 a, such as a home, may include an interface 120.The interface 120 can include any communication circuitry needed toallow a device to communicate on one or more links 101 with otherdevices in the network. For example, the interface 120 may include amodem 110, which may include transmitters and receivers used tocommunicate on the links 101 and with the local office 103. The modem110 may be, for example, a coaxial cable modem (for coaxial cable lines101), a fiber interface node (for fiber optic lines 101), twisted-pairtelephone modem, cellular telephone transceiver, satellite transceiver,local wi-fi router or access point, or any other desired modem device.Also, although only one modem is shown in FIG. 1, a plurality of modemsoperating in parallel may be implemented within the interface 120.Further, the interface 120 may include a gateway interface device 111.The modem 110 may be connected to, or be a part of, the gatewayinterface device 111. The gateway interface device 111 may be acomputing device that communicates with the modem(s) 110 to allow one ormore other devices in the premises 102 a, to communicate with the localoffice 103 and other devices beyond the local office 103. The gateway111 may be a set-top box (STB), digital video recorder (DVR), computerserver, or any other desired computing device. The gateway 111 may alsoinclude (not shown) local network interfaces to provide communicationsignals to requesting entities/devices in the premises 102 a, such asdisplay devices 112 (e.g., televisions), additional STBs or DVRs 113,personal computers 114, laptop computers 115, wireless devices 116(e.g., wireless routers, wireless laptops, notebooks, tablets andnetbooks, cordless phones (e.g., Digital Enhanced CordlessTelephone—DECT phones), mobile phones, mobile televisions, personaldigital assistants (PDA), etc.), landline phones 117 (e.g. Voice overInternet Protocol—VoIP phones), and any other desired devices. Examplesof the local network interfaces include Multimedia Over Coax Alliance(MoCA) interfaces, Ethernet interfaces, universal serial bus (USB)interfaces, wireless interfaces (e.g., IEEE 802.11, IEEE 802.15), analogtwisted pair interfaces, Bluetooth interfaces, and others.

FIG. 2 illustrates general hardware elements that can be used toimplement any of the various computing devices discussed herein. Thecomputing device 200 may include one or more processors 201, which mayexecute instructions of a computer program to perform any of thefeatures described herein. The instructions may be stored in any type ofcomputer-readable medium or memory, to configure the operation of theprocessor 201. For example, instructions may be stored in a read-onlymemory (ROM) 202, random access memory (RAM) 203, removable media 204,such as a Universal Serial Bus (USB) drive, compact disk (CD) or digitalversatile disk (DVD), floppy disk drive, or any other desired storagemedium. Instructions may also be stored in an attached (or internal)hard drive 205. The computing device 200 may include one or more outputdevices, such as a display 206 (e.g., an external television), and mayinclude one or more output device controllers 207, such as a videoprocessor. There may also be one or more user input devices 208, such asa remote control, keyboard, mouse, touch screen, microphone, etc. Thecomputing device 200 may also include one or more network interfaces,such as a network input/output (I/O) circuit 209 (e.g., a network card)to communicate with an external network 210. The network input/outputcircuit 209 may be a wired interface, wireless interface, or acombination of the two. In some embodiments, the network input/outputcircuit 209 may include a modem (e.g., a cable modem), and the externalnetwork 210 may include the communication links 101 discussed above, theexternal network 109, an in-home network, a provider's wireless,coaxial, fiber, or hybrid fiber/coaxial distribution system (e.g., aDOCSIS network), or any other desired network. Additionally, the devicemay include a location-detecting device, such as a global positioningsystem (GPS) microprocessor 211, which can be configured to receive andprocess global positioning signals and determine, with possibleassistance from an external server and antenna, a geographic position ofthe device.

FIG. 2 shows an example hardware configuration. Modifications may bemade to add, remove, combine, divide, etc., components as desired, andsome or all of the elements may be implemented using software.Additionally, the components illustrated may be implemented using basicdisplay devices and components, and the same components (e.g., processor201, ROM 202, user input/output devices 206, etc.) may be used toimplement any of the other display devices and components describedherein. For example, the various components herein may be implementedusing display devices having components such as a processor executingcomputer-executable instructions stored on a computer-readable medium(e.g., storage 205), as illustrated in FIG. 2.

Having described examples of network environments and contentconsumption devices that may be used in implementing various aspects ofthe disclosure, several examples illustrating how a display device maymonitor user actions during an advertisement, a display device mayrestrict a user's control of the display device during an advertisement,and efficacy file reports are created and used will now be described ingreater detail. The consumption device, which may be a user's tabletcomputer, personal computer, smartphone, DVR, etc., may monitor anyclient-side interaction with the user during an advertisement, such asdetecting a change in audio level or z-order of display elements. Aconfiguration file with such instructions may be developed by anadvertisement server and sent to a content server to store with anadvertisement that will be sent to the display device. In otherexamples, the display device may prohibit a user from muting anadvertisement during play.

FIGS. 3 and 4 illustrate different ways in which an advertisement may beimpaired during the streaming of video content. In particular, FIG. 3depicts a display 300, which may be a display screen on a computermonitor, tablet, television, smartphone, or other display device. Inthis example, a user may be using an Internet browser to view videocontent on a tablet computer. The browser may have a browser applicationdisplay area 301 (e.g., a panel or window) through which the userperuses the available video offerings from a video Internet site, andupon selecting on offering, the user may view the requested video in thebrowser application's display area 301 as a video stream, and at timesduring the consumption of the video content, the browser applicationdisplay area may display an advertisement 302. The browser applicationdisplay area 301 may also include a control bar, 303. This bar, 303,includes, but is not limited to, a pause button, 304, a play button,305, a progress bar, 306, and a sound control button (not shown). Thebrowser contains icons for moving backward in browser history, 307, andmoving forward in browser history, 308.

During display of the advertisement 302, the user may decide to open orswitch to a different application. For example, the user may open anInternet chat application to exchange messages in a conversation with afriend, and the chat application's display area 310 may partially ortotally obscure the browser application display area 301 and theadvertisement 302. In this example, while the advertisement, 302, isplaying during the streaming video content (as evidenced by the boldedplay button, 305), the user has accessed another application 310, and indoing so has obscured some of the advertisement 302 appearing in thebrowser application display area 301. Although this example obstructionis due to another application on the device, an obstruction can alsooccur from within the same application (e.g., if the user were to open amenu in the browser application).

The obstruction also need not obscure the advertisement 302 to be animpediment to the advertisement's effectiveness. For example, if theuser opened the chat application 310, and kept it on a different part ofthe display screen 300, without obscuring any portion of the browserapplication display area 300, the user's attention may still be divertedfrom the advertisement if the user were actively engaged in a chatduring the advertisement's duration. For example, if the chatapplication 310 received the display device's user input focus (e.g., ifthe user actively typed on a keyboard, sending the keystrokes to thechat application), it may be evident that the user's attention wasdiverted from the advertisement. Similarly, if the z-order of thebrowser display area 301 was changed so that it was at a lower level inthe order (lower being relative, in which higher z-order applicationdisplay areas overlap lower z-order application display areas). Changesin the application display area's size may also indicate a diverting ofattention (e.g., if the user were to shrink, or minimize, or hide, thebrowser application display area 301 during the advertisement 302). Theuser may also turn off the display function of the device, while leavingthe browser application operational on the device.

The FIG. 3 examples discussed above illustrates examples of a visualobstruction of the advertisement 302. In some situations, theadvertisement's audio may also be obscured by the user who, for example,may decide to turn down or mute the volume during the advertisement 302.FIG. 4 depicts impairment of an advertisement via the lessening of theaudio level during play, 400. In that example, an advertisement, 402, isimpaired such that the audio level has been lessened, as illustrated bythe lessening of the bars in the audio level box, 408, above the soundcontrol button, 407.

In both of these examples, the advertisement 302 was not given theuser's full and undivided attention, and as a result, the advertisement302 may have been less effective than expected. In embodiments herein,these impairments to the effectiveness of an advertisement may betracked and reported to the advertiser, and in some embodiments, theyimpairments themselves may be restricted.

As discussed above, this kind of obstruction may be problematic foradvertisers who pay to include their advertisements in video content. Assuch, although an advertisement, 302, plays during the video content,the user might not be watching the advertisement if he or she isaccessing another window, 310, which obscures the advertisement. Whilean advertiser has paid a content server to stream its advertisement,that advertisement might not be viewed. Therefore, in this example, anadvertiser may earn a refund credit for some or all of the funds paidfor showing the advertisement depending on which parts of theadvertisement were obscured and at what time.

An advertiser may decide that certain portions of the advertisement aremore important than others for a user to consume (e.g., the upper-leftcorner, the central area, the audio, the first five seconds, the lastfive seconds, etc.), and thus may negotiate for partial refund creditsdepending on which parts of the advertisement were obscured. Similarly,an advertiser may decide that a particular percentage of advertisementobscured yields a partial or total refund credit. These systems ofpartial credits may create a weighing system so that certain portions ofthe video, both physically and temporally, may be of more or less valueto an advertiser. For example, as in FIG. 3, an advertiser may decidethat only the bottom left portion of the screen, portion 313, must bevisible throughout the entirety of the advertisement, 302, in order toavoid a refund credit because that portion contains the advertiser'swebsite. In this case, the advertiser has weighed portion 313 moreheavily than the rest of the advertisement, and as such, would earn arefund credit if that portion were obscured. Alternatively, theadvertiser may decide that in order to avoid a refund credit, portion313 of the advertisement, 302 must be shown during the last five secondsof the advertisement, 302. In this case, a physical and temporal portionof the advertisement, 302, must be viewed in order for the advertiser tobe satisfied that its advertisement, 302, was effective.

Apart from earning refund credits, an advertiser may wish to understandwhich portions of its advertisement, 302, are obscured in order to gaugeuser interest in portions of its advertisement, 302. The system hereinmay provide data as to which portions of an advertisement were viewed atwhat time so that it can develop more effective advertisements based onuser viewing patterns. Compiling similar information from numerous userswould allow for a determination of efficacy that has not been accessiblepreviously. For example, if a user begins a chat program, which coversup most of the display area, but leaves exposed an animated area of theadvertisement, portion 313, the advertiser may determine that this user,and perhaps others, would enjoy more animated sequences inadvertisements. The advertiser may also determine that non-animatedsequences are less effective. Similarly, the systems herein may reportthe audio obstructions to allow an advertiser to know whether sound froman advertisement had impact on a user. For example, if a user obscuredan advertisement during the first few seconds of an advertisement butremoved the obstruction upon the sound of puppies barking, an advertisermay decide that baby animals, or the sound of baby animals, may helpimprove user interest in advertisements.

Further, the system herein may identify which applications or windowswere used to obscure the display area during the advertisement. Forexample, the system can note that the chat application obscured theadvertisement, and that other applications on the user's device did not.In such a case, an advertiser may be able to determine what a user findsmore interesting than an advertisement that is currently playing. Thisfeedback could be useful to an advertiser in that it determines userinterest in its advertisement compared to another program or window, andit also may assist in determining what users would prefer to see inadvertisements.

Using FIG. 3, an advertiser may wish to know that a user accessed a chatprogram, 310, during the advertisement. In that case, an advertisercould determine that a user is more interested in chatting with friendsthan watching an advertisement. Such information could be helpfulbecause an advertiser might create more advertisements showcasing socialsituations that cater more to people who chat frequently. Also, anadvertiser may determine that an advertisement is more or less effectivedepending on what kind of application is obscuring the display area. Forexample, an advertiser may determine that a user accessing a chatprogram is less distracted from the advertisement than a user whoaccesses his or her email client during the advertisement. This might bebecause emails take more concentration to craft than chats with friends.This information further could aid an advertiser in determining othermethods and avenues of advertising online. For example, an advertiserwith such feedback may decide to create advertisements for chat programsor may create interactive advertisements with which several peoplechatting can interact.

In addition to the above examples, the system herein may report what auser inputs into an obscuring program or window and the timing of thatinput. As discussed above in the example, an advertiser may determinethat a user is more or less distracted from an advertisement dependingon the program used to obscure the advertisement. In FIG. 3, the user isusing a chat program, 310, with which to chat with friends and has sentthe message, “Hello,” 311. An advertiser may be interested in whatmessages a user is sending to his or her friends during anadvertisement. For example, an advertiser may determine that userchatting with a friend about a product showcased in an advertisement isless distracted overall than a user chatting about something irrelevantto the advertisement. Also, an advertiser may determine that though verylittle of the display area is obscured by another window, a user is moredistracted from the advertisement if he or she is furiously typing to afriend throughout the entirety of the advertisement rather than typingslowly or pausing between messages. An advertiser may also test theefficacy of portions of an advertisement by determining at what timeduring an advertisement a user types to his or her friends in a chatwindow.

As discussed above, FIG. 4 illustrates a network browser, 401, whichincludes a display area, 412, playing an advertisement, 402, such thatthe audio level has been lessened during the advertisement from thelevel of the video content prior to the advertisement. In this example,while the advertisement, 402, is playing during the streaming videocontent (as evidenced by the bolded play button, 405), the user hasselected the sound control button, 407, and lessened the number of barsin the audio level box, 408.

Similar to the issues involved in obstructing the display area, 402, anadvertiser may wish to know at what time the audio level of itsadvertisement has been lessened. For example, an advertiser maynegotiate with the content server to accept a refund credit or a partialrefund credit if a user mutes the advertisement at any time. Thisreasoning may be due to the fact that an advertiser would find itunlikely that a user was effectively affected by the advertisementbecause he or she did not hear a portion of it. Similarly, an advertisermay wish to receive a partial refund credit if the user lessened theaudio level during part of the advertisement. For example, theadvertiser may receive a refund credit if the last five seconds of theadvertisement, 402, were played at a lower audio level because theadvertiser's name and product were explicitly stated during thoseseconds. An advertiser may also wish to receive refund credits based onthe percentage decrease in audio level by the user. In this case, theadvertiser may still receive refund credit without relying on the userto go to the extreme of muting the advertisement, 402.

Beyond earning credits, an advertiser may gather information as to theefficacy of its advertisement by observing how much the audio level islessened or increased and at what time. For example, if a user mutes anadvertisement during a song portion of the advertisement, an advertisermay decide that the song was ineffective. Also, if a user unmutes anadvertisement upon observing an animation, the advertiser may decidethat a user will likely wish to hear the accompanying sounds for ananimation. Further, an advertiser may hypothesize that a user lesseningthe audio level of an advertisement without muting it is mildlyinterested in the product advertised, more so than a user who mutes theadvertisement. The information gathered through these methods and thosediscussed in reference to FIG. 3 would be important to determine theefficacy of each advertisement.

Any of the methods mentioned above can be combined with one another tomeasure efficacy of an advertisement. For example, an advertiser maywish to determine at what times during an advertisement a user obstructsthe display area of an advertisement and whether those times correspondto times during which a user may lessen the audio level of theadvertisement. Measurements of this type may better determine userdistraction from the advertisement during play.

FIG. 5 illustrates an example process that a consumption device, such asa user's tablet computer, DVR, smartphone, personal computer, etc. mayemploy to monitor the efficacy of advertisements and send a reportcomprising the measurements of those parameters to the advertisementserver. The example consumption device below is a display device, suchas a personal computer or tablet. The FIG. 5 process may begin after theuser has requested to begin consuming content, such as after the userhas requested to begin watching an Internet video. Upon beginning theprocess, 501, the display device may load the requested video content(or the initial portion thereof), and may also load configurationinstructions 502. The configuration instructions may be downloaded aspart of a content download, and may contain various configurationsettings and data for the content delivery. The configuration data mayinclude information identifying a timeline for the content, andindicating what advertisements are scheduled to be presented, and atwhat playback times they are to be presented. The configurationinformation may also identify an address for one or more advertisingservers, which may be servers 106 or 107, that are to receive reports ofthe user's advertisement consumption behavior. These instructions allowthe device to determine which portions of the video content constituteadvertisements, and may also allow the display device to determine theviewing parameters to monitor for the advertisement, such as the levelof obstruction, audio level, z-order, etc.

Upon loading those instructions, the device must then determine if thevideo content has begun playing in the display area, 503. If the videocontent has not yet begun playing, the device will continue to checkuntil the video content begins to play in the display area. Afterward,the device will determine baseline parameters in Steps 504, 505, and506. These parameters comprise determining the audio level, 504, thez-order, 505, and the size of the application display area, 506.

In Step 507, the device determines whether the advertisement portion ofthe content presentation has begun. As noted above, this may bedetermined by checking the configuration information for the content.Alternatively, this may be determined by detecting an advertising markerin the streaming content. If it has not, the device recalculates thebaseline parameters to ready itself for the playing of theadvertisement. If the advertisement has begun, the device may retrievethe configuration file that it had previously loaded, 508. Thisconfiguration file comprises a set of instructions for the device toperform during the advertisement to create a report to send to theadvertisement server upon completion of the advertisement. Inalternative embodiments, the advertisement's configuration file may bedownloaded at a different time, such as prior to the content beginning,or even after the advertisement begins.

In the embodiment of FIG. 5, the device tests for particularuser-instigated changes that may occur during the advertisement. First,in Step 509, the device determines if the user has turned off thedisplay device or monitor. Turning off the display device may indicatethat the user did not observe the advertisement. In step 510, the devicemay then record information indicating whether the user turned thedisplay, along with the time of determination in a report file that willbe sent to the advertisement server.

In step 511, the device may determine whether the advertisement has beenobscured in is presentation. As noted above, an advertisement may beobscured in many ways including, but not limited to, another applicationvisually overlapping the advertisement, as illustrated in FIG. 3 anddescribed above. To determine whether an advertisement is obscured, thedevice may examine its own graphics systems to determine what images arebeing displayed on the areas of the screen, and whether the applicationcontaining the advertisement has any of its display area obscured. Ifany of that application's display area is overlaid by anotherapplication or object, the device may identify the portions of theadvertisement that are overlaid, and the item(s) and applications thatare overlaying those portions. As with the previous steps, theinformation identifying the obstruction of the advertisement and thetime of determination are saved in the report file in Step 512.

Next, in Step 513, the device determines if the audio level of the videocontent has changed during the advertisement. This determination mayretrieve a current audio level, and compare it to the audio leveldetermined in step 304 before the advertisement began. The device maystore information identifying this change in audio (including whethermuting was implemented) to the report file in step 514.

In step 515, device may then check to see if the size of the displayarea in which the advertisement is playing has changed from when it wasdetermined in step 506. This may include determining the new size and/orplacement on the screen (e.g., if the user repositioned theapplication's display area) of the application's display area, as wellas whether the application display area was completely minimized fromdisplay. Information identifying these sizes and/or changes In step 516,the device may store this data in the report file.

In step 517, the device may determine whether its orientation oracceleration has changed during the advertisement in Step 517. This maybe done, for example, by comparing its current orientation and/oracceleration data with previous orientation and/or acceleration data,which may have been obtained with step 506. The device may then writethat data and the time of measurement to the report file in Step 518.

In step 519, the device may determine if the user is inputtinginformation into another application on the device. This may be done,for example, by registering the device monitoring function with thedevice's user input capturing system, such as its keyboard, mouse, touchscreen and microphone systems, and receiving reports of the inputs beingprovided by the user. In step 520, the device may also identify theapplication that is receiving the user inputs (e.g., identifying thechat application), as well as the input the user is entering. The devicemay then write that information to the file along with the time ofdetermination in step 521.

After these measurement and any subsequent measurements are taken, thedevice may determine whether the advertisement has ended in step 522.The instructions as to the determination of whether the advertisementhas ended may be found in the instructions loaded with the video contentin step 502 (e.g., the configuration file may provide a time value forwhen the advertisement is over, or the device may examine an incomingstream for a marker identifying the end of the advertisement or thebeginning of a different portion of the content). If the ad is not over,then the device may return to step 509, repeating the variousmeasurements and data storage described above. Note that the comparisonswith the initially determined levels, z-order, display, etc., mayinstead be made against the previously stored version, if desired.

If, in step 522, the device has determined that the advertisement hasended, then the report file may be sent to the advertisement server instep 523 for further processing, which can include examining the reportto indicate how effective the advertisement was, and to determinewhether the advertiser is entitled to any refund or credits. Theadvertisement server may also assemble data to send a report to theadvertiser.

Once the report has been sent, the device may determine in step 524whether the application has completed presenting the content to theuser. This information may be contained in the instructions loaded instep 502 (e.g., as an expected duration), or the determination maysimply be made by determining whether an end-of-content signal has beenreceived, or if the content is no longer streaming. If the contentpresentation is not complete, then the device may return to step 504. Ifthe content presentation is complete, then the process may return to thestate prior to step 501, and await the next time the user requestscontent.

Note that FIG. 5 is merely an example. The steps of FIG. 5 may be added,removed, combined, divided, or rearranged to fit the needs of anadvertiser or display device. The same is true for the other examplesand processes described herein.

The FIG. 5 example was a passive example, in which the consumptiondevice simply monitored the user's behavior, and reported it. In otherembodiments, the consumption device may take a more active roleregarding the user's ad consumption behavior and options. FIGS. 6 and 7illustrate embodiments in which the display device plays an active rolein regulating a user's experience of an advertisement. FIG. 6 involvesan embodiment wherein the display device restricts a user's ability toobstruct an advertisement or lessen its audio level, 600. FIG. 7involves an embodiment wherein the device allows a user to make changesto the audio level, but then either pauses video content or activatesclosed captioning, 700.

FIG. 6 involves an embodiment of the method illustrated in FIG. 5wherein the display device restricts a user's actions during anadvertisement, 600. The method of FIG. 6 can be inserted anywhereapplicable in FIG. 5 and may replace any applicable steps of FIG. 5. Forexample, the method of FIG. 6 may replace steps 509-521 of FIG. 5. InStep 601, the display device may choose any or all of the possiblerestrictions on user behavior in the configuration file loaded in Step508, and may apply the restrictions throughout the duration of theadvertisement, or at specific portions of the advertisement as indicatedin the configuration data. For example, the advertiser may indicate inthe configuration file that some restrictions should be applied only tothe first ten seconds of the advertisement, while other restrictionsshould be applied only to the upper-left corner of the display area.

The display device may restrict the user's ability to change the size ofthe display area in step 602. The restrictions may include a totalprohibition on changing the display size, which would prevent the userfrom being able to reduce or minimize the size of the applicationdisplay area during the presentation of the advertisement. Therestrictions may also, or alternatively, place a minimum and/or maximumsize for the application display area. The user may be permitted toresize the display area to some degree, but not smaller than theminimum, or larger than the maximum. In some embodiments, the resizinglimitations may also identify whether the entire advertisement screen isto scale equally when the size is changed, or if a zooming effect ispermitted, and if zooming is permitted, then the limitations mayindicate a point in the advertisement that will be the central point ofthe zooming. So, for example, a user may be permitted to reduce the sizeof the advertisement, but in doing so the display will center and focuson the identified zooming central point (e.g., the central point of thead, or an upper-left quadrant). This control over the size of thedisplay area allows for the advertiser to ensure that particularportions of its advertisement are visible during play. Theserestrictions may also be temporal. The limitations may identify one ormore time segments of the advertisement (e.g., the first five seconds)during which resizing limitations are to be enforced.

In step 603, the device may restrict the user from changing the z-orderof the display area playing the advertisement. For example, the devicemay restrict the user from lowering the display area's z-order by morethan two units, or it may require that the advertisement display begiven the topmost level in the z-order, so that it is always on top inthe layered display. Alternatively, the device may prohibit changes inz-order, keeping the application at whatever level it was at for thepresentation of the non-advertising content. As with the previous step,these restrictions may vary with temporal portions of the advertisement.

In step 604, the device may restrict the user's ability to obstruct theadvertisement. Obstruction of the advertisement may include any or allof the methods discussed with regard to FIG. 3. The restrictions mayprohibit certain types of overlapping, in addition to (or aside from)the z-order restriction keeping the advertisement on top. For example,the restrictions may identify a portion of the advertisement (e.g., theupper-left corner, or the last five seconds) that cannot be overlaid byother application displays. The user may be permitted to move anotherdisplay on top of the advertisement, but the identified portion of theadvertisement would remain visible. Enforcement may include stopping themovement of another application's display window at the identifiedportion of the advertisement, or allowing movement of the otherapplication's display to overlap the advertisement's display area (butthe advertisement's identified portion would retain the top z-orderlevel and be visible, even if the other application's content isotherwise visible. As in step 604, the device may restrict obstructionof the advertisement at any temporal or physical portion of theadvertisement. For example, the device may restrict obstruction of theadvertisement during the last five seconds because the advertiser's nameand product are prominently presented.

The restrictions on obscuring may also depend on the obscuringapplication. For example, the advertiser may permit certain applicationsto overlay their advertisements, and prohibit others from doing so. Asan example, an advertiser may wish to encourage its viewers to access anInternet page for an advertised product. If the user, while viewing thead, decides to open a second browser application to the advertiser'sInternet site (which may be specified in the configuration data for theadvertisement), then the user may be permitted to move that browser'sdisplay area or window to overlap with the advertisement, and theadvertisement may allow the browser's content to obscure theadvertisement. In some embodiments, upon conclusion of theadvertisement, or if the user navigates the overlapping browser to adifferent web page that is not identified in the advertisement'sconfiguration data, the advertisement may resume the top z-orderdisplay, and move to appear atop the browser.

In step 605, the device restricts the user from lessening the audiolevel during an advertisement. In that step, the device may only allow auser to lessen the volume by two bars in the audio level box, or mayonly prohibit muting of the advertisement (and at which time aclose-captioned display may automatically appear atop the advertisementin response to the muting and without requiring the user to supply anadditional command for close-captioning).

Following these steps, the device may then write information collectedduring restriction to the report file to be sent to the advertisementserver in step 606. In this embodiment, the method would continue withstep 522 of FIG. 5. In the process above, the restrictions may be liftedin exchange for actions by the user. For example, a user who attempts tominimize an advertisement may be presented with a pop-up message,providing an option to allow the minimization if the user agrees to havea corresponding advertisement emailed to the user's email address (whichmay have been provided as part of an initial registration for service).

In FIG. 7, rather than restrict user action, the device implementsresponses to the user actions, 700. As with the method of FIG. 6, thesteps of FIG. 7 may be added to the embodiment of FIG. 5 or may replacecertain steps of FIG. 7. For example, the steps of FIG. 7 may replaceSteps 513 and 514 of FIG. 5. In Step 701, the display device determineswhether the user has lessened the audio level. If not, the method willcontinue to step 705 during which the device will write that informationas well as the time of determination to the report file. If the devicedetermines that the user has lessened the audio level, the device mayproceed to step 702 and perform either Step 703, 704, or both. In Step703, the device may pause video content upon lessening of audio level.In some embodiments, the device may resume playing the content (e.g.,the advertisement, although this may be applied to the other content aswell) in response to the user raising the audio level above apredetermined point, or returning the audio to the original level beforethe user turned it down. In others, the device may resume play if theuser presses play, without requiring an audio level change again. Instep 704, the device activates closed captioning upon lessening of theaudio level during an advertisement. Closed captioning might remain forthe entirety of the video content. Alternatively, the device might ceaseclosed captioning upon completion of the advertisement or if and whenthe user raises the audio level. In some embodiments, steps 703 or 704may only be activated if the audio level is lessened by more than aparticular amount or percentage. In other embodiments, those steps mayonly be activated upon muting the advertisement. Following steps 703 or704, the device may write the information and time of determinations tothe report file to be sent to the advertisement server.

FIGS. 8 and 9 illustrate additional embodiments of active methods inwhich a device can respond to a user's actions during an advertisement.FIG. 8, like FIG. 4, illustrates a network browser, 801, which includesa display area, 812, playing an advertisement, 802, such that the audiolevel has been lessened during the advertisement from the level of thevideo content prior to the advertisement, 800. In this example, whilethe advertisement, 802, is playing during the streaming video content(as evidenced by the bolded play button, 805), the user has selected thesound control button, 807, and lessened the number of bars in the audiolevel box, 808. In response to this lessening of audio level, the devicehas activated closed captioning, 813, during the advertisement, 802. Asdiscussed above, the closed captioning, 813, may last only for theduration of the advertisement, 802, or may disappear oncenon-advertisement content starts to play.

FIG. 9 illustrates an advertisement obstructed by another application,900. In this example, the advertisement, 902, is played on a networkbrowser, 901, inside a display area, 912, in the browser. The browsercontains icons for moving backward in browser history, 907, and movingforward in browser history, 908. The streaming video content, includingthe advertisement, 902, contains a control bar, 903. This bar, 903,includes, but is not limited to, a pause button, 904, a play button,905, a progress bar, 906, and a sound control button (not shown). Inthis example, while the advertisement, 902, is playing during thestreaming video content the user has accessed another application 910,and in doing so has obscured some of the display area, 912, of theadvertisement. In response to this obstruction, the device has pausedthe advertisement, 902 (as evidenced by the bolded pause button, 904).The device may unpause the content if the user removes the obstructingwindow, or the device may unpause if the user manually unpauses thecontent, among other possible methods.

The Steps discussed above may be added, removed or changed in any waythat satisfies the parameters of the advertisement server or displaydevice.

Figures l0a and 10b illustrate two different ways in which theconfiguration file of step 508 of FIG. 5 may manifest. Figure l0aembodies a pseudo-code manifestation of such a configuration file. FIG.10b embodies a table version of such a configuration file.

In FIG. 10a , the configuration file constitutes a function that runsduring the length of the advertisement. First, the variable “content”may be the number of seconds of content that have played at the currenttime. Then the variable, “audio level” may be the audio level of thecontent one second before the start of the advertisement. The variable“obscurity” may be the level of obscurity of the display showing theadvertisement, one second before the advertisement begins. The variable“z-order” may be the z-order of the display area one second before theadvertisement begins. The variable “ad content” is defined as the numberof seconds in the advertisement.

A void function “configuration” is then created. This function operatesa for loop dependent on the variable x. The loop runs from the start ofthe advertisement, x=0, to the end of the advertisement, x=ad content.During the loop, for each second, x, the function creates variablesassociated with measurements taken during the advertisement. Thefunction creates a variable “ad_audio_level” which may be the audiolevel of the advertisement at second x. The function creates thevariable “ad_obscurity” which may be the level of obscurity of thedisplay area during second x of the advertisement. The function alsocreates the variable “ad _z-order” which determines the z-order of thedisplay area playing the advertisement at second x.

Once those variables are set, the function determines whether ad audiolevel is less than audio level. If so, the function prints, “Userlessened volume at” x “seconds into the ad.” The function thendetermines if ad obscurity is greater than obscurity. If so, thefunction prints, “User obscured as at” x “seconds into the ad.” Finally,the function determines if ad _z-order is less than z-order. If so, thefunction prints, “User was engaged in a different window at” x “secondsinto the ad. This loop repeats for every second of the advertisement.

FIG. 10b constitutes a simple table version of the configuration file.The table simply requires the device to report a lessening of audiolevel, an obstruction of the advertisement, or a change in z-order ofthe advertisement and the time at which these actions occurred. WhileFIGS. 10a and 10b represent possible configuration files, they aresimplistic. Most configuration files produced with respect to thispatent will likely be more complex.

FIG. 11 illustrates an embodiment of the actions of the advertisementserver with respect to the display device's actions, 1100. Upon thestart of the method, 1101, the advertisement server may send advertisersoptions as to what parameters or measurements they can require a displaydevice to monitor, 1102. These parameters or measurements may encompassany of the steps discussed in reference to the previous Figures. Theserver would then receive a configuration file from the advertiserswhich includes those parameters or measurements, 1103. After that step,the server would likely store the configuration as part of theadvertisement it sends to the display device so that the device canactivate it upon receipt of the advertisement, 1104. After the serversends the content to the display device, Step 1105, the server thenawaits receipt of the advertisement effectiveness report file that thedisplay device develops after each advertisement played, 1106. Theserver then sends the feedback to advertisers so that the efficacy ofadvertisements can be studied by those advertisers, 1107, before themethod ends, 1108.

The FIG. 3 example illustrates one manner in which an advertisement maybe obscured on a device, such as a personal computer or tablet, that isable to support multiple simultaneous applications. However, as alsonoted above, secondary applications are not the only way in which anadvertisement may be obscured in its presentation. In some alternateembodiments, the features described herein may be used for a DVR' s orset-top box's presentation of audiovisual advertising content, such asthat received in a broadcast or multicast transmission of a videoprogram (e.g., the nightly news, a movie, etc.). In such embodiments,the DVR or set-top box (STB) may receive an incoming datastream carryingthe video program that the user wishes to view, and decode thedatastream to present the video program on an associated display device,such as a television or video monitor.

In such embodiments, the video stream carrying the video (e.g., anMPEG-2 transport stream) may also carry tags or codes (e.g., a uniqueadvertisement identifier) that may be inserted in the stream to indicatewhen commercials interrupt a video program, and when the video programcontent begins or resumes. The DVR or STB may be configured to respondto a commercial tag by monitoring the user's use of the DVR or STBduring the commercial's duration. As one example, the DVR or STB maydetermine whether the user changed channels during the presentation ofthe commercial. As another example, the DVR or STB may determine whetherthe user turned off the television, but not the DVR or STB, during thecommercial (e.g., this may be accomplished, for example, by exchangingmessages between the DVR or STB and the television, such as using HighDefinition Multimedia Interface (HDMI) signaling to indicate thepowering off of the television).

In some embodiments, the user may have multiple video sources connectedto the same video display device. For example, the user may have a videogame console and a DVR connected to a television, and during acommercial break the user may decide to stop watching the DVR programand start playing a video game. When the user switches the input sourceon the television, the television may be configured to report thatchange to the DVR (e.g., via the same HDMI signaling, or via a differentprotocol), and the DVR may note that the commercial on the video streamis not actually being viewed.

The tags or codes may be recorded as part of a DVR recording of thevideo program, and in that way, when the user plays back the recording,the DVR may once again detect the codes and track the commercial views.This feature can also be used when a viewer pauses and rewinds a lineartelevision stream (e.g., a live broadcast of a football game) to watch acommercial again—the user's DVR can track the repeated viewings of thesame commercial.

In some embodiments, the DVR may offer multiple tuners to the user,where each tuner is able to receive and decode a single stream forpresentation to the user. These tuners can allow the user to watch oneprogram and record one or more others (depending on the number of tunersavailable). When a user is viewing a first program using a first tuner,and a commercial begins, the user might switch to a different tuner towatch a different program and avoid the commercial. In some embodimentsherein, the DVR may track the user's switching of tuners, and may storeinformation indicating that the user did not actually view a commercialthat was received by the first tuner but not fully viewed by the user.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

1. A method comprising: determining, by a computing device, anadvertisement for output, wherein the advertisement is associated withat least one rule for modified output of the advertisement; causing, viaa display associated with the computing device, output of theadvertisement; receiving, during the output of the advertisement, arequest to modify the output; and causing, based on the rule and therequest, modified output of the advertisement.